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Background 



• February, 2005: Paris High Level Forum on Aid Effectiveness 

• More than 100 signatories— from donor and developing- 
country governments, multilateral donor agencies, regional 
development banks and international agencies— endorsed the 
Paris Declaration on Aid Effectiveness . 

• The Paris Declaration represented a broad consensus among 
the international community about how to make aid more 
effective. At its heart was the commitment to help 
developing-country governments formulate and implement 
their own national development plans, according to their own 
national priorities, using, wherever possible, their own 
planning and implementation systems. 



Agreed Principles 



Ownership: Developing countries must lead their own development policies and strategies, and manage 
their own development work on the ground. This is essential if aid is to contribute to truly sustainable 
development. Donors must support developing countries in building up their capacity to exercise this kind 
of leadership by strengthening local expertise, institutions and management systems. The target set by the 
Paris Declaration is for three-quarters of developing countries to have their own national development 
strategies by 2010. 

Alignment: Donors must line up their aid firmly behind the priorities outlined in developing countries 7 
national development strategies. Wherever possible, they must use local institutions and procedures for 
managing aid in order to build sustainable structures. In Paris, donors committed to make more use of 
developing countries' procedures for public financial management, accounting, auditing, procurement and 
monitoring. Where these systems are not strong enough to manage aid effectively, donors promised to 
help strengthen them. They also promised to improve the predictability of aid, to halve the amount of aid 
that is not disbursed in the year for which it is scheduled, and to continue to "untie" their aid from any 
obligation that it be spent on donor-country goods and services. 

Harmonisation: Donors must coordinate their development work better amongst themselves to avoid 
duplication and high transaction costs for poor countries. In the Paris Declaration, they committed to 
coordinate better at the country level to ease the strain on recipient governments, for example by 
reducing the large numbers of duplicative field missions. They agreed on a target of providing two-thirds 
of all their aid via so-called "programm based approaches" by 2010. This means aid is pooled in support of 
a particular strategy led by a recipient country— a national health plan for example— rather than 
fragmented into multiple individual projects. 

Managing for results: All parties in the aid relationship must place more focus on the end result of aid, the 
tangible difference it makes in poor people's lives. They must develop better tools and systems to measure 
this impact. The target set by the Paris Declaration is for a one-third reduction by 2010 in the proportion 
of developing countries without solid performance assessment frameworks to measure the impact of aid. 

Mutual accountability: Donors and developing countries must account more transparently to each other 
for their use of aid funds, and to their citizens and parliaments for the impact of their aid. The Paris 
Declaration says all countries must have procedures in place by 2010 to report back openly on their 
development results. 



Accra High Level Forum 



The Third High Level Forum on Aid Effectiveness (HLF-3) was held in Accra, Ghana from September 2-4, 2008 

The Accra meeting was different from its predecessors in that developing countries played a more active role in the 
preparations and the agenda. Some 80 developing countries took part in the regional preparatory events. Fifty-four 
developing countries participated in the OECD's 2008 Survey of progress against the Paris Declaration targets. Civil society is 
also increasingly involved in discussions of aid effectiveness; globally, more than 300 civil society groups, including grass 
roots groups, were involved in consultations in the lead-up to the Accra meeting. 

On the third day of the Forum, ministers endorsed the Accra Agenda for Action (AAA) . This ministerial statement has been 
developed with support from a multi-national consensus group working under the auspices of the OECD's Working Party on 
Aid Effectiveness . Attention is focused on stepping up progress towards the commitments outlined in the Paris Declaration 
by committing signatories to accelerating the pace of change by focusing on key areas that should enable them to meet the 
2010 targets agreed in Paris. Drawing on evidence from the latest evaluations, the 2006 and 2008 Surveys on Monitoring the 
Paris Declaration and on in-depth contributions from developing countries, the AAA identifies three main areas where 
progress towards reform is still too slow. 

1. Country ownership. The Accra Agenda for Action says developing-country governments still need to take stronger 
leadership of their own development policies and engage further with their parliaments and citizens in shaping them. 
Donors must commit to supporting them by respecting countries 7 priorities, investing in their human resources and 
institutions, making greater use of their systems to deliver aid, and further increasing the predictability of aid flows. 

2. Building more effective and inclusive partnerships. The Accra Agenda for Action aims to incorporate the contributions of 
all development players— middle-income countries, global funds, the private sector, civil society organisations— into more 
inclusive partnerships. The aim is for all the providers of aid to use the same principles and procedures, so that all their 
efforts are coherent and have greater impact on reducing poverty. 

3. Achieving development results— and openly accounting for them. The Accra Agenda for Action says the demonstration 
of impact must be placed more squarely at the heart of efforts to make aid more effective. There is a strong focus on helping 
developing countries to produce stronger national statistical and information systems to help them better monitor and 
evaluate impact. More than ever, citizens and taxpayers of all countries expect to see the tangible results of development 
efforts. In the AAA, developing countries commit to making their revenues, expenditures, budgets, procurements and audits 
public. Donors commit to disclosing regular and timely information on their aid flows. 
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Next Steps 



• The Accra Agenda for Action sets out a list of 
commitments for its signatories, building on 
those already agreed in the Paris Declaration. 
It asks the OECD's Working Party on Aid 
Effectiveness to continue monitoring progress 
on implementing the Paris Declaration and the 
Accra Agenda for Action and to report back to 
the Fourth High Level Forum on Aid 
Effectiveness in December 2011 in Seoul, 
Korea 



Why Evaluate the Paris Declaration? 



• Evaluation is part of the Paris Declaration and 
reflects its principles. 

• Accra Agenda for Action called specifically for an 
evaluation of the implementation, effects of PD. 

• Add value to the monitoring of commitments and 
feed into the High Level Forums in 2008 (phase I) 
and 2011 ( phase II synthesis). 

• What has been achieved and what has not - and 
why -will be a key questions at HLF 4. 



Phase II 'Architecture' 



Based on the principles of the Paris ■ 
Declaration: a joint evaluation comprising: 

• Country level evaluations assessing 
results/outcomes 

• Additional donor/agency HQ studies 

• Supplementary studies with focus on results 

• Synthesis report to be presented to 4th HLF 



Phase II Evaluation Model 



Two complementary foci: 

1) Implementation focus (expanded Phase I): How is the 
PD put into action? 

• More diverse set of countries + wider range of 
stakeholders 

• HQ studies by donors not included in Phase I 

2) Results focus: What has changed? 

• Identify PD influence on development effectiveness 
(outcomes & results) 
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Methodological Issues 



Challenges to evaluate the results: 

• Brief time span since PD endorsed (March 2005) 

• External influences make attribution difficult 

• Demanding data requirements 

To handle these challenges the evaluation should: 

• Have a balanced & sufficient sample of countries 

• Have sufficient coverage of sectors and themes 

• Produce information of good quality 

• Apply Complementary methods 

• Offer possibility of explanation and attribution 



Methodological Approach 



Therefore methods should include: 

• Synthesis of existing evaluations, research and 
indicators I 

• Comparative in-depth case studies chosen to contain a 
good cross-section of common themes/sectors 

• Longitudinal studies: backward looking to track PD-like 
(longer established) policies 

• Targeted studies to 'supplement 7 country-based 
evaluations 
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Managing the Evaluation 



Management Group responsible for: 

• Developing the overall evaluation framework and 

• Terms of Reference for Core Evaluation Team 

• Coordinating and managing the evaluation process 

• Guiding the component studies 

• Developing and managing supplementary studies 

• Producing synthesis of findings and recommendations 

• Dissemination 

• Evaluation Secretariat is at Danish Institute for 
International Studies 
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US Role 



• Member of International Reference Group (50 
countries) 

• Elected member of 6 country Core Management 
Group 

• Conduct of a US Donor Study via contractor 
(work started in January 2010) 

• US study is managed by Office of Director of US 
Foreign Assistance 

• US study reference group includes: F, State, 
USAID, USDA, HHS, Treasury, MCC 



Thank You 



Contact Information: 



Peter Davis 

Acting Coordinator, Office of Planning and Performance Management 

Office of the Director of Foreign Assistance I 

Department of State 
DavisPB@state.gov 
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Design for USG Paris Declaration 

Case Study 

Dr. Krishna Kumar 
Office of the Director of US Foreign Assistance 



Background 



• Generic "Term of Reference" for Donors 
Studies 

• TOR Focus on - Awareness, Commitment and 
Capacity Building in Headquarters 

• Although not comprehensive, TOR provided 
foundation to develop a framework for USG 
Evaluation 
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Challenges 



• USG foreign assistance is being administered by over 
twenty four departments and agencies. All these agencies 
and organizations have their own policies, strategies and 
programs. 

• While PD Evaluation exclusively focuses on development 
assistance, USG does not make a distinction between 
foreign and development assistance. 

• The five principles of Paris Declaration are normative and 
can be operationalized differently. This creates 
methodological problems. 
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Inter-Agency Focus 



• We designed the study as a multi-agency 
evaluation to capture the varying policies, 
strategies and programs of the USG 
departments/agencies. 

• We invited four USG departments/agencies which 
provide the largest bilateral foreign assistance. 
These are USAID, the State, HHR and MCC. 
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Criteria for Selecting Smaller Donor 

Departments 

• We also invited three additional agencies with relatively small 
portfolio of bilateral assistance, which meet the following criteria: 

• (a) Have disbursed at least $ 25 million of foreign assistance in FY 
2008 (last year for which data are available) 

• (b) Have designed and implemented long-term (minimum of an 
year duration) in-country assistance programs, and 

• (c) Be willing to provide access to documents, studies and 
personnel (as informants in interviews or surveys). 

• Three departments which offered to participate were Departments 
of Agriculture, Labor and Treasury. 
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Reference Group and Coordination 



• Reference group consisting of a representative from each of the 
above departments/agencies to guide the evaluation process. 

• Each member of the reference group serves as the main point of 
contact for PD evaluation. Responsibilities include (a) facilitating 
contractor's access to the documents, senior staff and officials of 
the concerned agency/organization, (b) reviewing the drafts of 
member's agency/organization case study and secure the approval 
or comments of member's agency, and (c) reviewing the synthesis 
report and provide detailed comments to the evaluation 
Secretariat. 

• The Office of the Director of Foreign Assistance took the 
responsibility for coordination. 
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Conceptualizing Foreign Assistance 



• Used the definition of foreign assistance given by 
Foreign Assistance Act of 1961, as amended: 

Foreign assistance ''...means any tangible or intangible 
item provided by the United States Government to a 
foreign country or international organization under this 
or any other Act, including but not limited to any 
training, service, or technical advice, any item of real, 
personal, or mixed property, any agricultural 
commodity, United States dollars, and any currencies 
of any foreign country which are owned by the United 
States." 
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Conceptualizing Foreign Assistance 



This definition is broader than the ODA definition 
which other donors use. However, it includes two 
major elements of ODA definition; first, the purpose is 
primarily to promote economic development and 
welfare in the recipient country and second the 
assistance should be given on a concessionary terms. 



• The advantage of using this conceptualization is that it 
is not only consistent with OMB and Congressional 
requirements but also better captures the assistance 
provided for governance, democracy promotion and 
security. 
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Multi-Case Study Design 



• Multi-Case Study Design; Separate case studies 
will be prepared for each participating 
department/organization. The findings of the 
seven case studies will be synthesized in a 
comprehensive report. 

• Both qualitative and quantitative methods will be 
used to gather data and information. These 
include review of documents, key informant 
interviews, and field surveys. 
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Thank You 



Contact Information: 



Dr. Krishna Kumar 

Senior Evaluation Advisor 
Office of the Director of Foreign Assistance 
Department of State ^^^H 
KumarK@state.gov 
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Paris Declaration 
Evaluation 

Prepared by: 
Social Impact, Inc 



L#1PACT 
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Scope of Work 

Commitment 

> Examine awareness, knowledge, and commitment 
of USG to the PD principles as well as the extent to 
which there are underlying factors shaping the 
development of effective strategies for 
implementing the PD 



Capacity 

> Extent to which the Knowledge, Commitment and 
Strategies, if any, developed by the USG have been 
transformed into operational principles and 
directives and staff capacity for implementation of 
the PD accords. 

> SI will differentiate between those practices that 
are consistent with PD principles, and those which 
are directly attributable to the PD. It will also 
identify those USG policies and practices which 
either constrain or impede the adoption of the 

SOCIAL IMPACT 



Scope of Work 



Incentives and Disincentives 

> Incentives and Disincentives for implementing 
the PD principles (Actual and perceived) 

General Discussion 

> Analysis of the underlying causal factors 
"which have or are likely to affect" USG 
implementation of the PD. 

> Lessons learned 



SOCIAL IMPACT 




Evaluation Design 
and Methods 



Mixed Method combining 

- Document review 

- Time series analysis of budget 

-Analysis of Agency program design and 
implementation procedures 

- Structured Key Informant Interviews 

- US Mission survey research 

- Site visits to 5 -6 US Missions 

SOCIAL IMPACT 



Case Studies 
and Team 



Conceptual Leadership: John Eriksson, 
Richard Blue and Ray Rist 

7 Case Studies 

- Department of State: Richard Blue/Kelly Heindel 

- USAID: Jerry Hyman/Mathias Kjaer 

- MCC: Jim Fox 

- HHS: George Grob 

- DOT: Jim Fox 

- DOL: Jack Sullivan/Kelly Heindel 

- USDA: Cindy Clapp-Wincek/Morgan Holmes 

- OMB, US Congress, NGOs, NSC: Jack Sullivan 2 

SOCIAL IMPACT 



USG Synthesis 
Report 



Comparative Analysis of Case 
Studies 

Integration of data from survey 
research and document review 

Analysis of Mission survey and 
interview data 

Report will highlight cross-agency 
findings, conclusions, and include 
policy and operational 
recommendations. 



SOCIAL IMPACT 



Time Frame 



Contract Award: 

Program Design and Methods paper: 
Background Research: 
Interviews: 
Site Visits: 
Mission Surveys: 
Case Studies completed: 
Synthesis completed: 
Presentation to OECD/DAC: 



Jan 27 2010 
Feb 9, 2010 
March 2010 
May to early June 
March through June 

June 
July 30, 2010 
Sept 15, 2010 
October 2010 
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Challenges 



PD is in the air, but not yet on the ground 

People in Washington are busy 

Transitional political environment (e.g. 
many political appointees have not been 
made yet) 

Comparing PD branded processes with 
"PD Like" processes. 

Dealing with very different bureaucratic 
cultures. 

Reducing bias % 

SOCIAL IMPACT 



Thank You 



Contact Information: 
Dr. Richard Blue 
Vice-President, Evaluation Services 

Social Impact 

richardblue@earthlink.net 
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